Status of operation data archiving system using Hadoop/HBase for J-PARC

Kikuzawa, Nobuhiro; Ikeda, Hiroshi; Kato, Yuko; Ouchi, Nobuo; Yoshii, Akinobu*

Proceedings of 10th International Workshop on Personal Computers and Particle Accelerator Controls (PCaPAC 2014) (Internet), p.193 - 195, 2015/02

http://epaper.kek.jp/PCaPAC2014/papers/fpo016.pdf

J-PARC (Japan Proton Accelerator Research Complex) consists of much equipment. In Linac and 3 GeV rapid cycling synchrotron ring (RCS), the data of over the 64,000 EPICS records for these equipment has been collected. The Data volume is about 2 TB in every year, and the stored total data volume is about 10 TB. The data have been being stored by a Relational Data Base (RDB) system using PostgreSQL, but it is not enough in availability, performance, and capability to increase of data volume flexibility. Hadoop/HBase, which is known as a distributed, scalable and big data store, has been proposed for our next-generation archive system to solve these problems. The test system was built and verified about data transition or database utilization. This report shows the current status of the new archive system, and its advantages and problems which have been obtained through our verification.